seo

Use ‘What Googlebot Sees’ to Find Scrapers

Most of you are aware I recently took a position at Tribune as their in-house SEO manager.

One of the first things I did was get a current state of the union.  After working through the issues of verification for the dozens of sites (another post, perhaps), I realized that ‘What Googlebot Sees’ for a handful of the domains is ‘Privacy Policy’.  Now, this is obviously a red flag for us to add a ‘nofollow’ tag on the link (that’s perhaps another post also), but in addition to that, I found something else.

What’s that?  What did I find?

Scrapers . . . lots of them.  You’d think they’d take the time to add a line of code to their bot to remove such a common piece of code as ‘Privacy Policy’, but . . . nope.

Take a peek at your Webmaster Central console under the ‘What Google Sees’ section.  Not only will it highlight what you should ‘nofollow’ but, if you dig a little deeper, you may find some scrapers.  What you do next is your decision.  Options? Take a legal stance, allow them to do it until you gain some more inbound links, reach out to them with an RSS feed (or other linking option), or do something a bit more blackhat, like making your ‘Privacy Policy’ type links an image with an alt tag for the targeted keyphrase.  😉  I’m not going to share what we’re doing (though it is not the latter), as that would lead to proprietary information; besides, that type of decision is going to be pretty site-specific, anyway.

P.S. They turned the river green on St. Patrick’s Day in Chicago.  Interesting, but not sure if it was worth standing in the cold for 45 minutes with my 4-year old son.  😉

Brent D. Payne

SEO Manager

Tribune Interactive

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button